Patient Engagement With and Perspectives on a Mobile Health Home Spirometry Intervention: Mixed Methods Study.

BACKGROUND: Patient engagement attrition in mobile health (mHealth) remote patient monitoring (RPM) programs decreases program benefits. Systemic disparities lead to inequities in RPM adoption and use. There is an urgent need to understand patients' experiences with RPM in the real world, especially for patients who have stopped using the programs, as addressing issues faced by patients can increase the value of mHealth for patients and subsequently decrease attrition. OBJECTIVE: This study sought to understand patient engagement and experiences in an RPM mHealth intervention in lung transplant recipients. METHODS: Between May 4, 2020, and November 1, 2022, a total of 601 lung transplant recipients were enrolled in an mHealth RPM intervention to monitor lung function. The predictors of patient engagement were evaluated using multivariable logistic and linear regression. Semistructured interviews were conducted with 6 of 39 patients who had engaged in the first month but stopped using the program, and common themes were identified. RESULTS: Patients who underwent transplant more than 1 year before enrollment in the program had 84% lower odds of engaging (odds ratio [OR] 0.16, 95% CI 0.07-0.35), 82% lower odds of submitting pulmonary function measurements (OR 0.18, 95% CI 0.09-0.33), and 78% lower odds of completing symptom checklists (OR 0.22, 95% CI 0.10-0.43). Patients whose primary language was not English had 78% lower odds of engaging compared to English speakers (OR 0.22, 95% CI 0.07-0.67). Interviews revealed 4 prominent themes: challenges with devices, communication breakdowns, a desire for more personal interactions and specific feedback with the care team about their results, understanding the purpose of the chat, and understanding how their data are used. CONCLUSIONS: Care delivery and patient experiences with RPM in lung transplant mHealth can be improved and made more equitable by tailoring outreach and enhancements toward non-English speakers and patients with a longer time between transplant and enrollment. Attention to designing programs to provide personalization through supplementary provider contact, education, and information transparency may decrease attrition rates.

Association of household net worth with healthcare costs after radical cystectomy using real-world data.

BACKGROUND: Financial toxicity of bladder cancer care may influence how patients utilize healthcare resources, from emergency department (ED) encounters to office visits. We aim to examine whether greater household net worth (HHNW) confers differential access to healthcare resources after radical cystectomy (RC). METHODS: This population-based cohort study examined the association between HHNW and healthcare utilization costs in the 90 days post-RC in commercially insured patients with bladder cancer. Costs accrued from the index hospitalization to 90 days after including health plan costs (HPC) and out-of-pocket costs (OPC). Multivariable logistic regression models were generated by encounter (acute inpatient, ED, outpatient, and office visit). RESULTS: A total of 141,903 patients were identified with HHNW categories near evenly distributed. Acute inpatient encounters incurred the greatest HPC and OPC. Office visits conferred the lowest HPC while ED visits had the lowest OPC. Black patients harbored increased odds of an acute inpatient encounter (OR 1.22, 95% CI 1.16-1.29) and ED encounter (OR 1.20, 95% CI 1.14-1.27) while Asian (OR 0.76, 95% CI 0.69-0.85) and Hispanic (OR 0.74, 95% CI 0.69-0.78, p < 0.001) patients had lower odds of an outpatient encounter, compared to White counterpart. Increasing HHNW was associated with decreasing odds of acute inpatient or ED encounters and greater odds of office visits. CONCLUSIONS: Lower HHNW conferred greater risk of costly inpatient encounters while greater HHNW had greater odds of less costly office visits, illustrating how financial flexibility fosters differences in healthcare utilization and lower costs. HHNW may serve as a proxy for financial flexibility and risk of financial hardship than income alone.

An Electronic Health Record-Based Automated Self-Rescheduling Tool to Improve Patient Access: Retrospective Cohort Study.

BACKGROUND: In many large health centers, patients face long appointment wait times and difficulties accessing care. Last-minute cancellations and patient no-shows leave unfilled slots in a clinician's schedule, exacerbating delays in care from poor access. The mismatch between the supply of outpatient appointments and patient demand has led health systems to adopt many tools and strategies to minimize appointment no-show rates and fill open slots left by patient cancellations. OBJECTIVE: We evaluated an electronic health record (EHR)-based self-scheduling tool, Fast Pass, at a large academic medical center to understand the impacts of the tool on the ability to fill cancelled appointment slots, patient access to earlier appointments, and clinical revenue from visits that may otherwise have gone unscheduled. METHODS: In this retrospective cohort study, we extracted Fast Pass appointment offers and scheduling data, including patient demographics, from the EHR between June 18, 2022, and March 9, 2023. We analyzed the outcomes of Fast Pass offers (accepted, declined, expired, and unavailable) and the outcomes of scheduled appointments resulting from accepted Fast Pass offers (completed, canceled, and no-show). We stratified outcomes based on appointment specialty. For each specialty, the patient service revenue from appointments filled by Fast Pass was calculated using the visit slots filled, the payer mix of the appointments, and the contribution margin by payer. RESULTS: From June 18 to March 9, 2023, there were a total of 60,660 Fast Pass offers sent to patients for 21,978 available appointments. Of these offers, 6603 (11%) were accepted across all departments, and 5399 (8.9%) visits were completed. Patients were seen a median (IQR) of 14 (4-33) days sooner for their appointments. In a multivariate logistic regression model with primary outcome Fast Pass offer acceptance, patients who were aged 65 years or older (vs 20-40 years; P=.005 odds ratio [OR] 0.86, 95% CI 0.78-0.96), other ethnicity (vs White; P<.001, OR 0.84, 95% CI 0.77-0.91), primarily Chinese speakers (P<.001; OR 0.62, 95% CI 0.49-0.79), and other language speakers (vs English speakers; P=.001; OR 0.71, 95% CI 0.57-0.87) were less likely to accept an offer. Fast Pass added 2576 patient service hours to the clinical schedule, with a median (IQR) of 251 (216-322) hours per month. The estimated value of physician fees from these visits scheduled through 9 months of Fast Pass scheduling in professional fees at our institution was US $3 million. CONCLUSIONS: Self-scheduling tools that provide patients with an opportunity to schedule into cancelled or unfilled appointment slots have the potential to improve patient access and efficiently capture additional revenue from filling unfilled slots. The demographics of the patients accepting these offers suggest that such digital tools may exacerbate inequities in access.

A Digital Health Intervention to Improve the Clinical Care of Inflammatory Bowel Disease Patients.

BACKGROUND: Inflammatory bowel disease (IBD) is a chronic condition that requires close monitoring. Digital health virtual care platforms can enable self-monitoring and allow providers to remotely surveil patients and efficiently identify those with active disease. OBJECTIVES: The primary aim was to design and implement an IBD remote monitoring program, identify predictors of patient engagement, and determine who found the chat to be a valuable tool. METHODS: We developed the IBD Virtual Care Chat, an electronic health record (EHR)-integrated chat to monitor electronic patient reported outcomes (ePROs), medication changes, and disease activity, and subsequently report concerning findings to providers via the EHR. All patients in the IBD practice over age 18 with a clinical encounter in the preceding 12 months were eligible to be enrolled. The primary aim was to identify predictors of patient engagement and determine who found the chat to be a valuable tool. RESULTS: Between May 2021 and March 2022, 2,934 patients were enrolled. A total of 1,160 engaged at least once and 687 (23.4%) continually engaged, submitting at least three ePROs. Disease severity (based on Harvey-Bradshaw Index or Simple Clinical Colitis Activity Index) did not impact ePRO submissions. Patients were significantly more likely to be continually engaged if they self-reported the presence of extraintestinal manifestations (7%, 95% confidence interval: 0.01-0.14; p = 0.04). Patient satisfaction remained moderately high with a median score of 8 (interquartile range: 5-10) on a scale of 1 (poor) to 10 (good). CONCLUSION: Our program demonstrates the potential for EHR-integrated digital health as part of routine IBD care to achieve sustained engagement with high patient satisfaction.

Impact of an automated peri-procedural digital health intervention on rates of emergency department visits and readmissions.

BACKGROUND: Providing timely peri-procedural education, reminders, and check-ins can improve patient adherence and clinical outcomes. We sought to retrospectively evaluate the impact of a peri-procedural digital health tool on emergency department (ED) visits and readmissions. METHODS: A digital health tool for peri-procedural care engaged patients at scheduled intervals, resulting in an overall engagement score. Multivariate models determined predictors of tool engagement and post-procedural 30- and 90-day rehospitalizations and ED visits. RESULTS: 11,737 unique completed procedures were analyzed from 10,438 patients. Patients of Black and Latinx race/ethnicity (vs White), those with Medicare and Medicaid insurance (vs commercial), and those with non-activated patient portals (vs activated) were less likely to engage. After adjustment for confounders, higher engagement with the tool was associated with lower rates of 30-day hospitalizations (OR 0.64), 90-day hospitalizations (OR 0.65), and 90-day ED visits (OR 0.77). CONCLUSIONS: Highly engaged patients had fewer 30-day and 90-day ED visit and readmissions, even after adjustment for key confounders. Engagement, and thus the resulting benefits, were not equitably distributed.

The Impact of Patient Travel Distance on Outpatient Visit Satisfaction: Comparison of Telehealth and Traditional In-Person Visits.

Introduction: Patient satisfaction has been shown to changes based on the distance a patient to see their physician. We sought to examine the effects of telehealth on patient satisfaction. Methods: We examined patient satisfaction survey scores from outpatient clinics at University of California, San Francisco. Patient home and clinic addresses were used to calculate distance in kilometers (km). Outcomes were "top scores (9-10)" and "low scores (<9)." Results: Of 103,124 evaluations that met inclusion criteria, those where patient traveled >100 km for in-person visits had more top scores (84%) than those traveled <10 km (80.2%). Relative to in-person visits, telehealth was associated with an increased odds (odds ratio [OR]: 1.48) of receiving a top score at all distances. Those traveling >100 km had the highest odds of top score for telehealth (OR: 1.86). Conclusions: Patients receiving care through telehealth, particularly those far from the outpatient clinic, are more likely to provide high patient satisfaction scores for the visit provider.

Digital Health at Enterprise Scale: Evaluation Framework for Selecting Patient-Facing Software in a Digital-First Health System.

The digital transformation of our health care system will require not only digitization of existing tools but also a redesign of our care delivery system and collaboration with digital partners. Traditional patient journeys are reactive to symptom presentation and delayed by health care system-centric scheduling, leading to poor experience and avoidable adverse outcomes. Patient journeys will be reimagined to a digital health pathway that seamlessly integrates various care experiences from telemedicine, remote monitoring, to in-person clinic visits. Through centering the care delivery around the patients, they can have more delightful experiences and enjoy the quality of standardized condition pathways and outcomes. To design and implement digital health pathways at scale, enterprise health care systems need to develop capabilities and partnerships in human-centered design, operational workflow, clinical content management, communication channels and mechanisms, reporting and analytics, standards-based integration, security and data management, and scalability. Using a human-centered design methodology, care pathways will be built upon an understanding of the unmet needs of the patients to have a more enjoyable experience of care with improved clinical outcomes. To power this digital care pathway, enterprises will choose to build or partner for clinical content management to operationalize up-to-date, best-in-class pathways. With this clinical engine, this digital solution will engage with patients through multimodal communication modalities, including written, audio, photo, or video, throughout the patient journey. Leadership teams will review reporting and analytics functions to track that the digital care pathways will be iterated to improve patient experience, clinical metrics, and operational efficiency. On the backend, standards-based integration will allow this system to be built in conjunction with the electronic medical record and other data systems to provide safe and efficient use of the digital care solution. For protecting patient information and compliance, a security and data management strategy is critical to derisking breeches and preserving privacy. Finally, a framework of technical scalability will allow digital care pathways to proliferate throughout the enterprise and support the entire patient population. This framework empowers enterprise health care systems to avoid collecting a fragmented series of one-off solutions but develop a sustainable concerted roadmap to the future of proactive intelligent patient care.

Design and implementation of a digital health home spirometry intervention for remote monitoring of lung transplant function.

BACKGROUND: We developed an automated, chat-based, digital health intervention using Bluetooth-enabled home spirometers to monitor for complications of lung transplantation in a real-world application. METHODS: A chat-based application prompted patients to perform home spirometry, enter their forced expiratory volume in 1 second (FEV1), answer symptom queries, and provided patient education. The program alerted patients and providers to substantial FEV1 decreases and concerning symptoms. Data was integrated into the electronic health record (EHR) system and dashboards were developed for program monitoring. RESULT: Between May 2020 and December 2021, 544 patients were invited to enroll, of whom 427 were invited remotely and 117 were enrolled in-person. 371 (68%) participated by submitting ≥1 FEV1 values. Overall engagement was high, with an average of 197 unique patients submitting FEV1 data per month. In-person enrollees submitted an average of 4.6 FEV1 values per month and responded to 55% of scheduled chats. Home and laboratory FEV1 values correlated closely (rho = 0.93). There was an average of 133 ± 59 FEV1 decline alerts and 59 ± 23 symptom alerts per month. 72% of patients accessed education modules, and the program had a high net promoter score (53) amongst users. CONCLUSIONS: We demonstrate that a novel, automated, chat-based, and EHR-integrated home spirometry intervention is well accepted, generates reliable assessments of graft function, and can deliver automated feedback and education resulting in moderately-high adherence rates. We found that in-person onboarding yields better engagement and adherence. Future work will aim to demonstrate the impact of remote care monitoring on early detection of lung transplant complications.

Impact of patient portal-based self-scheduling of diagnostic imaging studies on health disparities.

While many case studies have described the implementation of self-scheduling tools, which allow patients to schedule visits and imaging studies asynchronously online, none have explored the impact of self-scheduling on equitable access to care.1 Using an electronic health record patient portal, University of California San Francisco deployed a self-scheduling tool that allowed patients to self-schedule diagnostic imaging studies. We analyzed electronic health record data for the imaging modalities with the option to be self-scheduled from January 1, 2021 to September 1, 2021. We used descriptive statistics to compare demographic characteristics and created a multivariable logistic regression model to identify predictors of patient self-scheduling utilization. Among all active patient portal users, Latinx, Black/African American, and non-English speaking patients were less likely to self-schedule studies. Patients with Medi-Cal, California's Medicaid program, and Medicare insurance were also less likely to self-schedule when compared with commercially insured patients. Efforts to facilitate use of patient portal-based applications are necessary to increase equitability and decrease disparities in access.

Identifying Barriers to Successful Completion of Video Telemedicine Visits in Urology.

OBJECTIVE: The utilization of video telemedicine has dramatically increased due to the COVID-19 pandemic. However, significant social and technological barriers have led to disparities in access. We aimed to identify factors associated with patient inability to successfully initiate a video visit across a high-volume urologic practice. MATERIALS AND METHODS: Video visit completion rates and patient characteristics were extracted from the electronic medical record and linked with census-level socioeconomic data. Associations between video visit failure were identified using multivariate regression modeling and random forest ensemble classification modeling. RESULTS: Six thousand eighty six patients and their first video visits were analyzed. On multivariate logistic regression analysis, Hispanic or Latino patients (OR 0.52, 95%CI 0.31-0.89), patients insured by Medicare (OR 0.46, 95%CI 0.26-0.79) or Medicaid (OR 0.50, 95%CI 0.29-0.87), patients of low socioeconomic status (OR 0.98, 95%CI 0.98-0.99), patients with an un-activated MyChart patient portal (OR 0.43, 95%CI 0.29-0.62), and patients unconfirmed at appointment reminder (OR 0.68, 95%CI 0.48-0.96) were significantly associated with video visit failure. Patients with primary diagnosis category of men's health (OR 47.96, 95%CI 10.24-856.35), and lower urinary tract syndromes (OR 2.69, 95%CI 1.66-4.51) were significantly associated with video visit success. Random forest analyses identified insurance status and socioeconomic status as the top predictors of video visit failure. CONCLUSION: An analysis of a urology video telemedicine cohort reveals clinical and demographic disparities in video visit completion and priorities for future interventions to ensure equity of access. Our study further suggests that specific urologic indications may play a role in success or failure of video visits.

Patient Experience and Feedback After Using an Electronic Health Record-Integrated COVID-19 Symptom Checker: Survey Study.

BACKGROUND: Symptom checkers have been widely used during the COVID-19 pandemic to alleviate strain on health systems and offer patients a 24-7 self-service triage option. Although studies suggest that users may positively perceive web-based symptom checkers, no studies have quantified user feedback after use of an electronic health record-integrated COVID-19 symptom checker with self-scheduling functionality. OBJECTIVE: In this paper, we aimed to understand user experience, user satisfaction, and user-reported alternatives to the use of a COVID-19 symptom checker with self-triage and self-scheduling functionality. METHODS: We launched a patient-portal-based self-triage and self-scheduling tool in March 2020 for patients with COVID-19 symptoms, exposures, or questions. We made an optional, anonymous Qualtrics survey available to patients immediately after they completed the symptom checker. RESULTS: Between December 16, 2021, and March 28, 2022, there were 395 unique responses to the survey. Overall, the respondents reported high satisfaction across all demographics, with a median rating of 8 out of 10 and 288/395 (47.6%) of the respondents giving a rating of 9 or 10 out of 10. User satisfaction scores were not associated with any demographic factors. The most common user-reported alternatives had the web-based tool not been available were calling the COVID-19 telephone hotline and sending a patient-portal message to their physician for advice. The ability to schedule a test online was the most important symptom checker feature for the respondents. The most common categories of user feedback were regarding other COVID-19 services (eg, telephone hotline), policies, or procedures, and requesting additional features or functionality. CONCLUSIONS: This analysis suggests that COVID-19 symptom checkers with self-triage and self-scheduling functionality may have high overall user satisfaction, regardless of user demographics. By allowing users to self-triage and self-schedule tests and visits, tools such as this may prevent unnecessary calls and messages to clinicians. Individual feedback suggested that the user experience for this type of tool is highly dependent on the organization's operational workflows for COVID-19 testing and care. This study provides insight for the implementation and improvement of COVID-19 symptom checkers to ensure high user satisfaction.

Utilization patterns and efficiency gains from use of a fully EHR-integrated COVID-19 self-triage and self-scheduling tool: a retrospective analysis.

OBJECTIVE: Symptom checkers can help address high demand for SARS-CoV2 (COVID-19) testing and care by providing patients with self-service access to triage recommendations. However, health systems may be hesitant to invest in these tools, as their associated efficiency gains have not been studied. We aimed to quantify the operational efficiency gains associated with use of an online COVID-19 symptom checker as an alternative to a telephone hotline. METHODS: In our health system, ambulatory patients can either use an online symptom checker or a telephone hotline to be triaged and connected to COVID-19 care. We performed a retrospective analysis of adults who used either method between October 20, 2021 and January 10, 2022, using call logs, electronic health record data, and local wages to calculate labor costs. RESULTS: Of the 15 549 total COVID-19 triage encounters, 1820 (11.7%) used only the telephone hotline and 13 729 (88.3%) used the symptom checker. Only 271 (2%) of the patients who used the symptom checker also called the hotline. Hotline encounters required more clinician time compared to those involving the symptom checker (17.8 vs 0.4 min/encounter), resulting in higher average labor costs ($24.21 vs $0.55 per encounter). The symptom checker resulted in over 4200 clinician labor hours saved. CONCLUSION: When given the option, most patients completed COVID-19 triage and visit scheduling online, resulting in substantial efficiency gains. These benefits may encourage health system investment in such tools.

Patient engagement in a mobile health intervention to improve preparedness for prostate biopsy.

OBJECTIVE: We designed and implemented a peri-procedural text message (SMS) program for patients undergoing transrectal prostate biopsy and aimed to evaluate predictors of patient enrollment and engagement with the SMS program. METHODS: We designed an SMS-based program with 8 messages containing web-based modules with educational content and reminders confirming MRI for fusion biopsy, antibiotic adherence, enema use, and anticoagulation cessation. Data on patient demographics, enrollment, and engagement with modules were collected from June 1, 2018 to February 28, 2021. Engagement was defined as a patient clicking a link delivered via SMS to access modules. We made multivariable models to identify predictors of patient enrollment and engagement. RESULTS: Of the 1,760 prostate biopsies between June 2018 and March 2021, 1,383 (78.6%) were enrolled in SMS, 182 (10.3%) in email, 106 (6.0%) in both, and 240 (13.6%) were not enrolled. Of 1418 patients enrolled, 1,270 (89.6%) engaged with at least one module. African American patients had 50% lower odds of being enrolled (OR = 0.50, 95% CI 0.28-0.96; P = 0.03), but once enrolled there were no differences in engagement. Patients for whom English was not listed as their primary language had 60% lower odds of engagement (OR = 0.40, 95% CI 0.17-1.00, P = .04) and patients who were single or divorced had a 40% lower odds of engagement (OR = 0.60, 95% CI 0.41-0.91, P = 0.01). CONCLUSIONS: A cohort of older men undergoing prostate biopsy were able to engage with a text message-based education and reminder program. Future efforts must address barriers to enrollment for Black or African American men and improve accessibility to non-English speaking patients.

Are We Reaching Everyone? A Cross-Sectional Study of Telehealth Inequity in the COVID-19 Pandemic in an Urban Academic Pediatric Primary Care Clinic.

The COVID-19 (coronavirus disease 2019) pandemic brought rapid expansion of pediatric telehealth to maintain patient access to care while decreasing COVID-19 community spread. We designed a retrospective, serial, cross-sectional study to investigate if telehealth implementation at an academic pediatric practice led to disparities in health care access. Significant differences were found in pre-COVID-19 versus during COVID-19 patient demographics. Patients seen during COVID-19 were more likely to be younger, White/Caucasian or Asian, English speaking, and have private insurance. They were less likely to be Black/African American or Latinx and request interpreters. Age was the only significant difference in patient demographics between in-person and telehealth visits during COVID-19. A multivariate regression showed older age as a significant positive predictor of having a video visit and public insurance as a significant negative predictor. Our study demonstrates telehealth disparities based on insurance existed at our clinic as did inequities in who was seen before versus during COVID-19.

Improving natural language information extraction from cancer pathology reports using transfer learning and zero-shot string similarity.

OBJECTIVE: We develop natural language processing (NLP) methods capable of accurately classifying tumor attributes from pathology reports given minimal labeled examples. Our hierarchical cancer to cancer transfer (HCTC) and zero-shot string similarity (ZSS) methods are designed to exploit shared information between cancers and auxiliary class features, respectively, to boost performance using enriched annotations which give both location-based information and document level labels for each pathology report. MATERIALS AND METHODS: Our data consists of 250 pathology reports each for kidney, colon, and lung cancer from 2002 to 2019 from a single institution (UCSF). For each report, we classified 5 attributes: procedure, tumor location, histology, grade, and presence of lymphovascular invasion. We develop novel NLP techniques involving transfer learning and string similarity trained on enriched annotations. We compare HCTC and ZSS methods to the state-of-the-art including conventional machine learning methods as well as deep learning methods. RESULTS: For our HCTC method, we see an improvement of up to 0.1 micro-F1 score and 0.04 macro-F1 averaged across cancer and applicable attributes. For our ZSS method, we see an improvement of up to 0.26 micro-F1 and 0.23 macro-F1 averaged across cancer and applicable attributes. These comparisons are made after adjusting training data sizes to correct for the 20% increase in annotation time for enriched annotations compared to ordinary annotations. CONCLUSIONS: Methods based on transfer learning across cancers and augmenting information methods with string similarity priors can significantly reduce the amount of labeled data needed for accurate information extraction from pathology reports.

Interactive Data Visualization Tool for Patient-Centered Decision Making in Kidney Cancer.

PURPOSE: Patients and providers often lack clinical decision tools to enable effective shared decision making. This is especially true in the rapidly changing therapeutic landscape of metastatic kidney cancer. Using the International Metastatic Renal Cell Carcinoma Database Consortium (IMDC) criteria, a validated risk prediction tool for patients with metastatic renal cell carcinoma, we created and user-tested a novel interactive visualization for clinical use. METHODS: An interactive visualization depicting IMDC criteria was created, with the final version including data for more than 4,500 patients. Usability testing was performed with nonmedical lay-users and medical oncology fellow physicians. Subjects used the tool to calculate median survival times based on IMDC criteria. User confidence was surveyed. An iterative user feedback implementation cycle was completed and informed revision of the tool. RESULTS: The tool is available at CloViz-IMDC. Initially, 400 lay-users and 15 physicians completed clinical scenarios and surveys. Cumulative accuracy across scenarios was higher for physicians than lay-users (84% v 74%; P = .03). Eighty-three percent of lay-users and 87% of physicians thought the tool became intuitive with use. Sixty-eight percent of lay-users wanted to use the tool clinically compared with 87% of physicians. After revisions, the updated tool was user-tested with 100 lay-users and 15 physicians. Physicians, but not lay-users, showed significant improvement in accuracy in the updated version of the tool (90% v 67%; P = .008). Seventy-two percent of lay-users and 93% of physicians wanted to use the updated tool in a clinical setting. CONCLUSION: A graphical method of interacting with a validated nomogram provides prognosis results that can be used by nonmedical lay-users and physicians, and has the potential for expanded use across many clinical conditions.

Supervised line attention for tumor attribute classification from pathology reports: Higher performance with less data.

OBJECTIVE: We aim to build an accurate machine learning-based system for classifying tumor attributes from cancer pathology reports in the presence of a small amount of annotated data, motivated by the expensive and time-consuming nature of pathology report annotation. An enriched labeling scheme that includes the location of relevant information along with the final label is used along with a corresponding hierarchical method for classifying reports that leverages these enriched annotations. MATERIALS AND METHODS: Our data consists of 250 colon cancer and 250 kidney cancer pathology reports from 2002 to 2019 at the University of California, San Francisco. For each report, we classify attributes such as procedure performed, tumor grade, and tumor site. For each attribute and document, an annotator trained by an oncologist labeled both the value of that attribute as well as the specific lines in the document that indicated the value. We develop a model that uses these enriched annotations that first predicts the relevant lines of the document, then predicts the final value given the predicted lines. We compare our model to multiple state-of-the-art methods for classifying tumor attributes from pathology reports. RESULTS: Our results show that across colon and kidney cancers and varying training set sizes, our hierarchical method consistently outperforms state-of-the-art methods. Furthermore, performance comparable to these methods can be achieved with approximately half the amount of labeled data. CONCLUSION: Document annotations that are enriched with location information are shown to greatly increase the sample efficiency of machine learning methods for classifying attributes of pathology reports.